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Abstract 

We prove that if the outer billiard map around a plane oval is 
algebraically integrable in a certain non-degenerate sense then the 
oval is an ellipse. 

In this note, an outer billiard table is a compact convex domain in the 
plane bounded by an oval (closed smooth strictly convex curve) C. Pick 
a point x outside of C. There are two tangent lines from x to C; choose 
one of them, say, the right one from the view-point of x, and reflect x in 
the tangency point. One obtains a new point, y, and the transformation 
T : x \ — > y is the outer (a.k.a. dual) billiard map. We refer to [31 HI [5] for 
surveys of outer billiards. 

If C is an ellipse then the map T possesses a 1-parameter family of invari- 
ant curves, the homothetic ellipses; these invariant curves foliate the exterior 
of C. Conjecturally, if an outer neighborhood of an oval C is foliated by the 
invariant curves of the outer billiard map then C is an ellipse - this is an 
outer version of the famous Birkhoff conjecture concerning the conventional, 
inner billiards. 

In this note we show that ellipses are rigid in a much more restrictive 
sense of algebraically integrable outer billiards; see [2] for the case of inner 
billiards. 

We make the following assumptions. Let f(x,y) be a (non-homogeneous) 
real polynomial such that zero is its non-singular value and C is a component 
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of the zero level curve. Thus / is the defining polynomial of the curve C, 
and if a polynomial vanishes on C then it is a multiple of /. Assume that a 
neighborhood of C is foliated by invariant curves of the outer billiard map 
T, and this foliation is algebraic in the sense that its leaves are components 
of the level curves of a real polynomial F(x, y). Since C itself is an invariant 
curve, we assume that F(x, y) = on C, and that dF is not identically 
zero on C. Thus F(x,y) = g(x,y)f(x,y) where g(x,y) is a polynomial, not 
identically zero on C. Under these assumptions, our result is as follows. 

Theorem 1 C is an ellipse. 

Proof. Consider the tangent vector field v = F y d/dx — F x d/dy (the 
symplectic gradient) along C. This vector field is non-zero (except, possibly, 
a finite number of points) and tangent to C. The tangent line to C at point 
(x, y) is given by (x + eF y , y — eF x ), and the condition that F is T-invariant 
means that the function 

F(x + eF y ,y-eF x ) (1) 

is even in e for all (x, y) G C . Expand in a series in e; the first non-trivial 
condition is cubic in e: 

W(F) .= F xxx F y — 3F xxy F y F x + 3F xyy FyF x — F yyy F x = (2) 

on C. We claim that this already implies that C is an ellipse. The idea is 
that otherwise the complex curve / = would have an inflection point, in 
contradiction with identity (J2J). 
Consider the polynomial 

H(F)=det( p % p p p^p p )■ 

\ yy x x y yi xx y x y x J 

Lemma 2 One has: 

1) . v(H(F)) = W{F); 

2) . H(F) = H(gf) = g s H(f) on C; 

3) . If C is a non-singular algebraic curve with a defining polynomial f(x,y) 
and (x, y) is an inflection point of C then H(f)(x,y) = 0. 
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Proof of Lemma [2J. The first two claims follow from straightforward com- 
putations. To prove the third, note that H(f) is the second order term in e 
of the Taylor expansion of the function f(x + ef y , y — ef x ) (cf. hence 
H(f) = at an inflection point. □ 

It follows from Lemma [2] and (J5J) that H(F) = const on C. Since C is 
convex, H(F) ^ 0, and we may assume that H(F) — 1 on C. It follows that 
g 3 H(f) — 1 vanishes on C and hence 



where h(x, y) is some polynomial. 

Now consider the situation in CP 2 . We continue to use the notation 
C for the complex algebraic curve given by the homogenized polynomial 
f(x : y : z) — f(x/z,y/z). Unless C is a conic, this curve has inflection 
points (not necessarily real). Let d be the degree of C. 

Lemma 3 Not all the inflections of C lie on the line at infinity 
Proof of Lemma [31 Consider the Hessian curve given by 



The intersection points of the curve C with its Hessian curve are the inflection 
points of C (recall that C is non-singular). The degree of the Hessian curve is 
3(d — 2), and by the Bezout theorem, the total number of inflections, counted 
with multiplicities, is 3d(d — 2). Furthermore, the order of intersection equals 
the order of the respective inflection and does not exceed d — 2, see, e.g, 
[6]. The number of intersection points of C with a line equals d, hence the 
inflection points of C that lie on a fixed line contribute, at most, d(d — 2) to 
the total of 3d(d — 2). The remaining inflection point lie off this line. □ 

To conclude the proof of Theorem [TJ consider a finite inflection point of 
C. According to Lemma (2J at such a point point, we have / = H(f) = 
which contradicts (j3J). This is proves that C is a conic. □ 



Sf 



■ 3 H(f) -l = hf 



(3) 
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Remarks. 1). It would be interesting to remove the non-degeneracy as- 
sumptions in Theorem [TJ 

2). A more general version of Birkhoff's integrability conjecture is as 
follows. Let C be a plane oval whose outer neighborhood is foliated by 
closed curves. For a tangent line t to C, the intersections with the leaves of 
the foliation define a local involution a on I. Assume that, for every tangent 
line, the involution a is projective. Conjecturally, then C is an ellipse and the 
foliation consists of ellipses that form a pencil (that is, share four - real or 
complex - common points). For a pencil of conies, the respective involutions 
are projective: this is a Desargues theorem, see []]. It would be interesting 
to establish an algebraic version of this conjecture. 
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